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Attention is focussed on, the estimation of the KR20 reliability coefficient 
when data are incomplete. In terms of point estiinate, it is found that the 
analysis of variance procedure (Henderson's Method 1) would be most suitable 
'among thje three techniques under study. Confidence coefficients are provided 
for .the KR20 when data dre Incomplete; " ^ / \ , 
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* ESTimriON OF THE K20 i^ELI/f^lLITY 
COEFFICI-ENT WHEfl DATA ARE JNCttMPLETE 



^ ^ i' » •!. Introduction 

The calculation of a reliability of a test such as the Kuder-Richardson 
coefficient twenty (KR20) or. its more genral counterpart alpha [Cronba'di, 
1951} frequently assumes that data are cofnplete. In other words each exam- 
inee is expected to respond to every item. Yet data completion is rarely - 
realized in ma^ny situations. For known or untold reasorfsT^ariy students 
choose to skip a difficult item rather than to contemplate a wild or edu- 
cated guess when penalties are imppsed for theincorrect responses. If the 
proportion of missing responses is fairly smafl,.one may estiiftate these 
responses and then apply an appropriate formula to the conplete set of data. 
vrner> testing is construed as tha realization of a two-way factorial design' 
.[Lord, 1955; Kris^of, 1963, 1970; Feldt, 1965], various techniques^ des- 
cribecl in such texts as Cochran and Cox [1966] or Winer [1971] may be called 
upon^. The extent to which the estimated rgsponses would bias the ;:omputed 
reliability^ however, does not seem to have been fully explored. 'Hence it 
appears desirable to base reliability computations directly on the available 
-data '^nd- to disregard totally the miss^ng-fesponses, 

^ ^ "tI^ /present study aims at the exploration of several festimatton proce- • 
durel for test reliability when data are incomplete,. Attention will be fo- 
.cWsed' On the KR20 >ndex (e.g. with test 'items scored as 0 or Ij, Conceiv- 



ably ^he VesultSwjItould be 'expected to hold for the'. more qeneral ind^x alpha 
since the 0 - 1 scaring presents the most serious violatton on the assumpittoi 
of the iipdel used' in the sfejdy. It will be assnnned on the part of the reader 
farailian'ty with tfie corftent of Chapter 7-of lord and Hovick (1968), . / . 



tif\e linear model [Searl, 1971] and the work of Feldt [1965] m\ mo(teling 
testing as random effect,' two-way factorial design* 



2. Description of three Estimation ' , 

■ . Procedures for the KR20 Coefficient 

U will now be assujned that the response of subject i (i = a) on 

item j (j ^ l/:..^ b) may pe represented as y . • = p + a. + B- + e. ., where '.'^^ 

V constarit-,.a. - L{0,a^2), gj 1(0,0^2) and e^^. L(0,ag-21_and r 

all random 'Variable? are independent. The Jiotation L("0, o^) represents any 

distributior> with 0 and^^ariance a^. It -is not required that this 

distribution is normal > Given the various parameters as defined, the population 

reliability index is pon- ba ^ / (bo ^ + a . ^ ^ 

dO q a * € 

The estimation'^ on the basis of sample test data may be Sosnewhat , 
facilitated by considering the estimation of <; ^ and a J- separately. Thua a 
reasonable estimate r2Q of pgQ may be. arrived at by replacing and o^^ re- 
•spectively By twp sqltably chosen unbiased estimates s^^ and' s^^^ This in no 
way guarantees that rgQ^ would average out at p2q> ho-^ever uhbiasedne*ss should be 
expected to hold at^^least asymptotically under nliTd* regularity gDnditions.* 
Under normality , 'it is known [Kristof, 1970] ^at an unbiased estimate for the 
population reliability may be given by.ck linear 'function oV .the sample 
reliability. Granting that the pattern of missing data^ is fixed from sample, to 
sample, there are at least thr^e ways to estimate both xx^ and a^.' All methods 



a • 



yield unbiased statistics for a ^ and! c^^. and reduce to-^.the traditional 

a e ' , , • . , ' ^ 

variance component estimation when dataware c6n^lete. * ^ ' 

The first two procedures, namely the analysis of var^iance (AtiOVA) and , 
fitting. constantsMFITCO) techniques are due to i^enderson [195331. The^ are 
also referred to as *Henders(^n*s Method 1 BndlHendersOn^^s Hethpd 3. Details 
about these methods along with their rationale^ may tie fojthd in^Se^rl [19^^^ , 
Chapter 10]. .Basically, Af^OVA attem})ts t^ extertd the fomulatfon of the ^ urn 



V of sqliares in a ^^fanced' design to the case of unbalanced data. Thus three 
soijrces of variation ere i denti lyi ed , naroely the total variation SS^^^, the 
row*(bbj^} variatiorr SS^ and .the (item) column variation SSg [See T^ble^l 
for the computational procedures.] The unbiased estimates s ^ and s'^^trill 
be found among the line^ cop*i nations of these yriations. Henderson's 
Method 3-(FITC0), on/the other h.and, focuses or/the -equation y.. = y + a. + B- 

+ as a linear -Inodel , considering (in tys study) the rcj^ variation SS^^ 

as' the. sum of squai^ explained by the^^s after being adjusted for the effects 

,of the g.'s and v [S^e Table 2]. Unbiased estimates for a ^ and a ^ may be 

J . a e ^ 

found as previously described. 

The third procedure, due to Koch [1968], Vs called the sywnstric sums 
(SYSUM) method. It is based on the synii^tlric sums of squared differences of 
the form (y.j - y^,y)^. Three symmetric sums [refered to as h^, hg.and 
. in Table 1] are there linearly combined to Wrm the unbiased estimates s ^ and* 

Q 

S 2 • / ' ■ • 

r . ..^ n' 

^ f 

. ' ^ Tables 1 and 2 



It may be noted that both procedures AMOVA and SYSUH are computationally 
simple and can be implemented easily by hand. The FITCO method is more cornplex, 
involving^ the inversion of an usually large matrix {of order of b - 1). Of 
course this should not present any problem where computer facilities are avail- 
able. . ' " ' ' ' 

' To provide the readers with some 'feeling about the three described methods,, 
'a ni^erital exartiple is presented in Table 3. ' ■ ^ 



Table 3 



3. Locjic of the Simulations 
When normality is assumed for the distributions described early In Section 2, 



and when the^pattem of missing <lata 4s fixed in advance ." the sampling variances 
of s^2-art;d-Sg2 may&e computed [Searl', 1971]. Bush and Anderson [1963] stadied 
several cases of the two-way design with planned unbalancedness. The general 
trend which may be deduced from their results is 'that when is much larger 
than <j^2^ the Af^OVA estimates have smaller variance than the FITCO estimates, 
but otherwise the FlTCO estimates are less varisble. Thus> in estimating test 
reliability p^q with, fixed {hence, kna^n) pattern of missihq^ata, it s'egms 
th^t the WJOy^\£rocedure would provide a mqre stable ^estimate r^^ than the 
FITCO does when q^q is fairly Iw. 'Conversely, .estimates based on the FITCO 
techniqi^ wotjTd show less fluctuation than those deduced from the ANOVA when 
is high. . • , 

^ In mental testing, it seems likely that the pattern of missing responses 
is'function of the ability level of the examinees-. When guessing is not^a 
major factor,* it would be reasonable to assise for exarrole that examinees with 
lower ability wiTl leave more items untouched than examinees with hjgher abil- 
. Ity. For these situatidns the. variahce calculations displayed in Searl [1971, . 
Chapter 11] and the findings of Bush and Anderson as stated would not automat- 
ically holdj - . ' ' ' 

. It may>e noted that the model y^jj = ji a- + +"ev . .implies that 
e^ch respon's.e y^ ; takes - at. least - fjcftJr separate values. " The lower bound is * 
occasioned when all random variables a-, and e.. have positive probatil 1 ties 
at oTily two identicaV points. .The estina^tion procedures previously described for 
P2Q are th^rgfore applicable as l6ng as jfehere are at least four response cate- 
gories for each test item. T-fiis ,requ1|a^nt is satisfied for data collected 



from Sjcales o^f the Likert type. Th^O - 1 scoHna procedure, on the other hand, 
clearly yfolates the functional^ form y.- = p + o. + 'B. +.e,.. ScHJie-data reported 
In Feldt [1965^/ however, in4icate .that £his linear mo'del may still serve as 
arf'aproximation for reliability^ studies iw.heri the §et of ?) - 'l responses, is • ' 



Even under normality 'any theoretical computation of the ?amp55ng van'- • 
ance of each of the three estimates rgQ would involve ^Wful efforts when the 
pattern of missing data is let to vary from sample to sample. With 0-1 data 
'such a computation may not be possible unles^ of course one^.is prepared to rely 
on an exhaustive enumeration GiveQ the computer time constraint on, the part 
of the author, it seans appropriate to resgrt to Honte Carlo simulations >tio 
compare the -performance of the AffcVA. FITCO and SYSUM procedures. The next- 
section vrill present the design of the first simulation based on artificial data. 

4. Oes,tqn of Simulation .Study 1 
'> * ^ 

For reason of mathematicar tractafaility, the beta-binomial model'was 
chosen- to generate the test responses in the first -simulation. This model assumes 
that (i) each exaiainee with true ability e responds 'independently tp the it€ms 
with'8 being the probability of obtaining a correct answer; and 
(ii) the true ability's for the population of examinees follows a beta 

density of the fonn ^ ■ . . > 

^(V) = — L— - 1 (1 - e)^<- ^ ■ ■ . 

B(u,v) 

where u > o and v >o. ^ ^ ^ 

With\the beta-binomial framework so stated, the population reliability for a' 

^ test consisting of b items is known to be p- = b/(b + u + V). 

20 

It se^ms reasonable to postulate that the proportion of missing response 

is a nonincre^sing function of the ability of the examinee. C^ne woujd as5uroe 

, of course that guessirrg is niyt a serious factor.) A missina response function' ^ 

Di(e) willliave to be specified. Though the set of nonincreasing functiojis is -, /' 

infinite, .a linear form was chosen for m{e) for reason of simplicity. In V' * 
' ' •* ^ 

^ % * ' ' .\ 

other words, the p^roportlon of missinp(^esponses is' . , / ' - 

• • m(e') = ce* + d 



where c and d arre two suitaWy chosen constants." Over the total population 
of examinees, the expected value for in{9) may be 'noted to be 

■ E mCe) = cu/ (u + v) + d. ■ 
If p denotes the" overall proportio^of missing responses, then 

cu/ (u + v) +d = p. . " * - . • 

To get a second equation so that'c and d may be solved, it was a'ssiimed that 

' ■ . . ' V 

examinees with highest ability (e = 1) would not miss any item. This implied 
that c + d = 0, and hence , • * 

m{9) = p { u + v) (1 - e)/v. • . . - 

.With b; being the number of items, an exajninee with true ability 6 wdUld be 
expected to'skip»bm(e> i\eras. Of course this number had to be rounded to 
the nearest integer in the study. It may be noted that though only one 
.tffousand sa/nples Were generated in this study, 'the overall pbserved proportion 
of missing responses is almost identical to the posted value p* 

As the first step in t>ie simirlation process j the percentile points of the 
true' abiUty distribution were computed via the IB?^ subroutine BDTR [19713. 
The IK-I subrouting RAfiDU [1971] was then used to (i) generate a random sample 
of a true ability values, (ii) generate at each true ability 6 the b 0 - 1 
responses, and (iii) generate at each true ability 9 the pattern pf the bm(9} 
missing responses. These three steps generated an incomplete matrix of 
responses* The^ three estimation procedures AfiOVA, FIT^O and SYSUM were then, 
applied to obtain th^ sample ^2Q^" whole process Was repeated one thou- 
sand times to est^imate the means ^20'^ corresponding mean square 
errors (MSE) which were used to c^cfihpare th^perfdnnance of the estimation - 
procedures under a variety of situations* ^ 

5* Results of Simulation Study 1 
Table. .4 reports the means and mean square errors of the three samplet 
estimates Tgo'^ for eighteen situations with a = 4(\and b = 20. The pppu-' 



lation rel-iabili.ty' wasj chosen to be^p^Q = .5 (low), ,7 (rroderate) and .9 ^' 
'(high). For each reliability leyel .'three tru^ ability distributions were 
selected to represent severa-l levels of skewness y. Finally^ except for 

» * 

one case.^the proportion of. missing data was choseR as 20% (moderate) and 
40% (high). * ■ - . ' 

The following trends may be inferred 'from the dclfa ort Table 4.*' 

(a) Under' all situations under" consideration, bdth the ANOVA and FITCO 
provide estimates which behave alnost identi^lly in terms of bias and . 
Tnean square error. " « - - 

(b) For syiiinetric '(true ability or ^est score) dis^^^utions, the three 
iftethods are about equally efficient. Though the SYSUM estimate tends to 
show -more bias or mare variability than the other tWo 'estimates 5 the 
discrepancies are not subs.lantial . 

(c) For skewed distributions, the SYSUM estimate tends to display'more pro- 
nounced bias or variability than the other two estimates. 

<d)" Other .things being equa), the means of both th§ WMA and FITCO estU % 
mates relate positively to the skewness level of the true ability or 
test score distributions. The reverse trend holds for- the SYSUtl 
estimate. 

{e) As pxpected, the three estimates perform better when the propprtion^of 
missing responses is smaller. f 



Table 4 



These findings indicate clearly^ that for all practical purposes, the 
tflOVA estimate, is the most suitable in dealing with incomplete data^ at 
least with those conformirvg to ,the b^ta-binomial modek This^ estimate is 
fairly simply to compute an'd tends to show less bias and less variability 



• • , ■ . ■■ ■ ■ . . . 

than the FITCO (a computational nightmare) and SYSUH estimates, inmost 
situations. . - , A ^ ^ 

• * ♦ 

6. Siq^ulation 'Study 2 

While the beta-binomial model was chosen mainly ffir its mathematical 
4;ractability, most real life test data rarely conform to any simple distri- 
bution. It would seem desiratle to replicate some of the conclusions af 
Section. 5 using some data of this type. 4 

A second simulation was conducted via random sampling (without re- 

plapementj from three data sets, ti) Data 1 consist of the response^ of • 

> 

532 examirrees to a 138 item test. The test score distribution bas mean 

76.75, standard deviation 27.09, skewness .259 and reliability .964. Using 

■ ■ ■ /* 

the Speanran-BfOwn formula for projection, i't was found that a test consisting 
of k = 20 i4:eihs of Data 1 would have a KR20. index of .796. (ii) Dat? Z'~ ■ 
is a subset of Data 1, consisting of the. responses of 428 examinees. The 
test score d^ribution" has jnean 63.67, standard deviation 17.77, -skewnes^ 
.096 and ^e^ia^^l^ty .909.^ Tlje projected reliability of a 20-item test is 
P20 = -590. (iii^) Finally Data 3 is a subset of Da-ta 2 with ^314 examinees. 
The test score tdistribution has 'mean ^55.23, staridard deviation 12.17, skew- 
ness -.271 and ^reliability .805. The projected reliability for a 20-item . 

* * 4 ♦ 

test is p^Q = ^375. 

For the pcfsent simulati^, the missing response function was choaen to 
be a^ decreasing* linear function of the test, score obtained from the 138-item 
test for each examinee'. As in Section 4, it was assumed that examinees '"vnth 
a score of 138 would not miss any item. • . ' 

The results of the second simulation study are 'reported in Table 5. They 
indicate clearly tfrat the AHOVA and FITCO estimates' behave* aTfho'st identical in- 
terms of bias and'mean square error. in all circumstances under investigation.^ 
The SYSUH estimate tends to show more bias and/or more variability than the 



" ' ^ ' • . -9- . • • - . 

other two estimates. These observations reinforce the major ^conclusion of 
^ the- first simuUtion Study that'tbt. A:I0VA "procedure seems to bk most suita- 
ye in dealing with .incomplete data. * 



Table 5 • ^ . ' 
* 



• 7, Confidence Interval for the KR2Q < - . .• ; • 

* • Reliabjlity Based on Incomplete Data . 

^ . „ ■ ' • 

Under normality, confidence Intervals for the KR20 reliability may be - 

^^stabjished when data ^re complete [Feldt, 1965], An extension will be oade 
""-to the case of incomplete data. •' , 

Referring to the notations of Table 1 and 2 it may be noted that under 
normality, the Various sums of sq^uares are distributed as chj-squares up to a' 
multiplicative constant.- More specifically, it is well known that (i) the*'. 
..adjusted sum of squares due to A, SS. = R{u, a, b) - T, is distributed -as 
■ [h + (a _ 'i)a^2j _ ij. ^i,-) ^^e erroV sum 6f squares SSg^ = • - . 
Tq «» is distributed iS,'Ka^\^{f)\ and (iii)' the tv/o chi-squares 

are i'ndependent. It follows that the ratio \ 

. • • ■ < 
X ? f ss./ss_ •. • 

is distribjuted a§ - • . * - / 

' ' ' [h a - 1] F{a - l,f} • V 

v/here F represents .an F variable with listed degrees of freedom. Algerbraic 
manipulations "yield 

s 2 . 

A = h —2— + a - 1 ' 

• s 2 . . » . • 



■ • ''20 

= h ; ; + a' - 1 

b(l - .rgV . ^ • 

ate 

where s^, s^ anct hence Tg^^^omputed v4a the JITCO procedure. Let 100(1 - a)> 



be gi-ven confidence level and let 



B (a - l.f)' = T . . (f, a -1) 

1 U 1 u « 



^ ■ ■ 



Then a 100 



)(1 - a')i conffdence interval for .the ratio a ^/a ^ is Wven by,t&e' 
4^ - I , ,ae 



two limits • . ,1 .if 

., ^ = (VL^,- a + l)/h- , . • ■ - • ^ 

and . • • ; . • : * , 

• » . ' 

SinQg^P2Q = <^g^) morKjtonically increasing v/ith o'^/o^^; the " 

corresponding 100(1 - 0)% conficience interval of p2(j is given by (pyj,P2) where 

and « ■ . ' . 

iNuperical example . Granting that the normal theoT7 can be applied to the, 

r 

data of Table 3, an 80% confidence interval was found to be (.029 - .629) Tor 
a ^/oj^, TfTe corresponding 80% confidfenceHjiterval for p^q was dediv:ed *as 
C.145 - .791-). . . , • . . ^' . ; ' • 

To conclude this section, ^the following remarks may be made, 
(a) The^ data' coTnpi led by Te*ldt [1965] ^and those of Jabl« 4 with y 

indicate that thfe normal mo<lelJ[iords up fairly welVfor symmetric 
\ : distributions based on 0 - 1 Responses. ,0ne should hope at^ least 
that^e procedures previously, descrified would prdvide an ?pproxi- 
mate confidenfce intel^al for when^the test ^core distribution 
is^ot baj^ly skewed. The Jacknife procedure as implemented by 
Pandley and. •Hubert [19/5] may be* .adapted to the case* of incomplete ' 
. *, data if normality a^^ars to be a botiierin? ass^wfsption/ 




(b) The data s/mulated in this study clearly sHW that the estimates 
. ^sed on Af^OVA and FITCQ are almost identical tii terms of mean .and 
" mean ^square, error. This \ndicates that in case^"bf -need, olie may • 
,go ahead and use.^the AJOVA^rgQ to est^lish-a" confidence intervil' 
for P2Q. ' This wou-ld grfeatTy simplify ±he* computations, 

! ' ^ ' ^ . 

• ' / . 8, Concl us ion _ . . ' ? ' 

Attention >/as focused in the study on the estimation of th^'KR20 ^lia- 

bility coefficient when data are incomplete. In terms of point estimation. It 

* " * * 

was found that the analysis of variance procedur^would be most suit^le' among 
,the three techniques under study. , Other techniques are of course available. 
But by and large they flo not seem to be easily implenfen-^d for test data*. One 
of them is the minimum normed 'quadratic unbiased estimation (MINQUE)^proj)osed 
by Rao [1973f ^Iso other references listed in this text], Undei* normality, ^ 
MIHQUE provides locally minimum variance unbiased quadratic estimates for 
bo4:h and o^^. ^ Unfortunately MIffQUE at the present time requires inversion 
of large matri^s cfnd therefore may not be computationally 'justified in most 
testing situations. ' ' . 

Finally confidence intervals based' on normality are provided. The mass 
of data on the effect of nonnormality on the F test and the forties of the 
Central Limit Theoreih should^ convince the readers that the procedure so des- 
cribed" would provide at least an approximation for a nurrfcer of testing situ- 
ations. ^ , ' ' / 



. " • Table 1 • " -I. 

Computing Formulae for the ANOVA, | 
' and SySUM Variance Estimates* :| 

' . ' - . i 



Model: y^. = ^ +'a. + a.-.,- 

1 = l,***,a; j - l,...,b; With n. . = 0 for empty cells %id n . • =^ 1 for 
n.on-empty cells; N = E E n..; ' % 

i j , 

Analysiis of variance estimators 
Calculate 

2 , - 2 



and 



5 -.^^ 



2 . . 2 



ki = £ n. • /f/ ■ • ki = E n . fW '" 

' i ^' j ."^ 

At ='{N - kp/{N -b) ■ and^ X2 = {Jl - kp/CN*- a). . ^ 



' Then 



s = ^ . 1^ ' * ' 

^ , N - k' - k' + 1 



K = [T. -. Tr - (N - b)s^lr(i^ - bj . 



- and 

Syfiietric 'Sums .estiijiatorS 

_ .CalCMlate k^ = Mk^ ajjd = flkj . . 

j 



and '-^ 



Table 1 . 

{contd.) • ' . 

. ■ > , • 

,2 - 2 

Then , ' s = b.^ - hp and = h. + ho - h^o 
a Aa B e A B AB 



* Adapted from Sedrlin971, Chap-ter 11]. It should be noted that the two 
expressions for h^ and hg on page 4S8j^of Searlemust be interchanged. TTi 
dot notation refers to summation. 
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: ■ > • Table 1^ ' ' ' . \ 



'* '^^"^ Computing Formulae for the FITtt) 

^ ' ' Variance Estimates* ' #1 



Coniputatlon for R(u', a,' S-) , ■. . ■ - 

Comsute $or j and j ''=!,».. ,b-1 

c . . = n . E n. ./n. . 
. a • 

' a _ 

' - ' J " ^i i^i 
whe^ is the mean of the ifh >cw. 
Let C = {c..,} and r. = -{rj 
' Th^n^ K { u, a, 6) ='J^ + r'C"|^^|>. . 

■ Fi-ttinq xnpnstants method estimators 

Denote " h- = N - "b • «y 
and ■ f = N a --^s+'l." 



Then 
and 



s! = [T„ - R(v, a,'i)3/f 



= CR<y, - Tg - (a-l)Sg3/h . 



* Adapted f rem Sear! CT971, Chapter 1]].' 



■ ■ ; Table- 3 - 

* , .A Nunerical Example qf 
the AfiOVA, FjrcO, and- SysUM Estimates 
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6 


1 • 


■ K ^ 


P 




X 


' 1 


0 ■ 


1 ' 


0 


' 0 


1 1, 



Coding: 1 = /Correct response 
0 =- incofrj^t response 
X = nrissin'g response 



RetiabiHty ANOVA = ..615 
■ estimate FITCO = .547 
SYSUH = .663 
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Table 4 



^feans (and Mean .Square Errors) of tbe-Af^OVA, FITCO and SYSUM Reliability H- * 
Estirpates Based- on Data Simulated from the Beta-Binomial ' 

Model 





• 

Population parameters 


p 


Means (KSE) of r2Q based on ^ 

AfwvA FITCO ' $ys\)K 

• 

e 



u = 
Y = 


6.000 
14.000 


Y = 

V 

P = 

20 


.385. , 
.500 


■20% 
402 


u = 
V = 


10.000 
10.000 


T = 
o — 

20 


/ooo 


20% 


Y = 


14.0"^ 
6.009" 


Y = 

P = 
20 


-.386 
.500 


20% 

m 

c 

- V 


U = 
Y = 


2.571 
6.000 


Y =' 

P J" 
20 


.519 
.700 


• 20% 
40% 


U = 
Y = 
• 


4.286 
4.286 


Y = 

P = 
20 


.000 
.700 


20% 
40% 


.u = 

V = 


6.000 
2.571 


Y = 

'-^^ 


-.519 
.700 


20% 
4(3% 


U- r 
Y = 


'.667 
1.556 


Y = 

P = 
20 


.141 
.900 


20% 
40% ' 


u =. 

Y = 


1.111 

•uin 


/ Y = 

P = 
20 


gooo 

.900 


20% 
40% 


u = 
V = 


1.556 
.667 


Y = 

r p = 


-.141 
.900 


20% 
30%* 



.533(.in) .534(.191) .508(.2T0) 

.549(.207) .550(.208) .489(.264) 

.504(.M8) .504(.149) ,515(.153) 

.491 (.196) ' .4f3{.196) .520(.210) 

.485(.15d) - -.485(.l"51) ' .564(.130) 

.472(.ia0) .471(.182) .64-3(-128) 

.7lO(.097), •.710(.097^: .68:^.112) 

.708(.116) .708{.118)" .637(.180) 

.692(.082) ,. .692{.082) .698(.084) 

,685(.107) .685(.108) .710(.n2) 



.671 (.090) ■.670{.091) .736( 
.655(.lll) .653(.111> .790( 



.071 f 
.062j 



20 



.898(.030) .898(.030) .888(.035) 

.898(.&34) .899(,034) .875(.056) 

.895(.027) .895(.027) .897(.085) 

.883(.032) .883(.0324 . .900(.0^1) 

.832(.032) .881 (.032) . ■ .910(.024) 

.862(.036) .860(.036) .913(.024) 



■ The stfnulation failed at p = 40% due to some negative values for di(6). 
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Table 5 



Aean (and .Mean Square -Errors) of the AfiOVA, .Frg:0 and srm Reliability 
Estimates' Based on Randcmi Samples, from Three Sets' of Real Data 



Popul a ti on . Parameters 



No. 



20 



Heans(HSE) of f^^ based on 



AfKJVA ' . - 



FITCO 



SYSUH 



"1 



.259- . .796 



2. .096 

3. -.271 



.590 
.375 



.20% 



20% 
40% 



20% 
.40% 



.795 (.05^) 
S 795 (.052) 

.552(.123) 
.544(.159) 

.331 (.176) 
.321 (.204) 



.7^5(.057) 
.793(.061) 

.553(.123) 
.547(.154,) 

r 

.329(.173) 
.322(.202) 



.815(.053) 
.843(.048) 

.553(.127) 
.528(.17U 

.318(.176) 
>295(.205) 
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